I also wonder whether it would be helpful to turn language experiments into a single packaged script (train tokenizer + adapt model + possibly run eval?) So that it is easier to onboard and have the others run experiments.

Might be in the future, but at least during these sprints, let's keep it separate.

We might also want to reconfigure the file structure?

My thoughts would be something like:

multilingual-modeling/
- lang-adapt/
    - README.md
    - scripts/
    - finetune/
        - *.py
    -  *.py
- evaluation/
    - eval_xnli/
    - eval_exp_sentence_retreival_eval/

Yea the structure is a mess right now. There's too many duplication (e.g., on the eval side, we actually don't need eval_xnli) due to legacy codes before.

I am working on it right now.

1fb6504

multilingual-modeling/
- lang-adapt/
    - README.md
    - scripts/
    -  *.py
- evaluation/
    - wikiann/  #scripts
    - xnli/  #scripts
    - eval.py
    - README.md
- exp_sentence_retreival_eval/

for now.

@lintangsutawika What do you have in mind in the finetune/ folder?

this makes sense to me, but I had problems downloading XNLI when there was a folder called "xnli" in the same path. Renaming to anything else (xnli_scripts, etc) fixes this problem.

@haileyschoelkopf Fixed by a8486d4 (using scripts_*) instead.

@yongzx I'm not sure. I think parameter-efficient finetuning should be included in lang-adapt/